Main Analysis

The following barcharts plot means with 95% confidence intervals. This first figure plots the overall rate of belief, click and sharing outcomes for stimuli (true and false).

Tables are the results of mixed effect models run as specified in the pre-analysis plan with linear regressions. We use mixed effect models because we are not interested in the “effect” of each specific respondent/headline on the outcome (belief, click, share), rather we hope our results generalize to other headlines and respondents. However, we know that the outcomes from the same respondent and for the same headline are likely correlated, so we want to control for that by including headline and respondent random effects.

For robustness, we also present results using logit models. These tables present odds ratios (odds ratios > 1 indicate a positive relationship, < 1 indicates a negative relationship). In all models, the emotion and veracity (true/false) variables have been recoded such that true = .5, false = -.5 and emotion = .5, absence of that emotion = -.5.

Neutral or any emotion

Linear regression mixed effects model

Dependent variable:
belief click share
(1) (2) (3)
neutralC -0.172*** -0.206*** -0.224***
(0.029) (0.019) (0.019)
trueC 0.263*** 0.071 0.148*
(0.090) (0.066) (0.079)
neutralC:trueC 0.112* -0.032 0.026
(0.057) (0.037) (0.035)
Constant 0.699*** 0.719*** 0.580***
(0.045) (0.033) (0.040)
Observations 13,410 13,409 13,409
Log Likelihood -5,996.359 -5,329.677 -6,794.762
Akaike Inf. Crit. 12,010.720 10,677.350 13,607.520
Bayesian Inf. Crit. 12,078.250 10,744.890 13,675.060
Note: p<0.1; p<0.05; p<0.01

Logistic mixed effects model

  belief click share
Predictors Odds Ratios p Odds Ratios p Odds Ratios p
(Intercept) 4.21 <0.001 4.92 <0.001 1.96 0.010
neutralC 0.30 <0.001 0.22 <0.001 0.23 <0.001
trueC 7.54 <0.001 1.96 0.121 2.96 0.031
neutralC * trueC 0.85 0.541 0.51 0.005 0.92 0.695
Random Effects
σ2 3.29 3.29 3.29
τ00 1.44 respID 2.37 respID 3.48 respID
0.77 headline 0.46 headline 0.63 headline
τ11 0.11 headline.neutralC 0.10 headline.neutralC 0.05 headline.neutralC
ρ01 0.01 headline -0.68 headline -0.80 headline
ICC 0.40 0.47 0.56
N 1341 respID 1341 respID 1341 respID
10 headline 10 headline 10 headline
Observations 13410 13409 13409
Marginal R2 / Conditional R2 0.186 / 0.515 0.070 / 0.510 0.068 / 0.592

Anger

Linear regression mixed effects model

Dependent variable:
belief click share
(1) (2) (3)
angerC -0.121*** -0.097*** -0.135***
(0.046) (0.036) (0.039)
trueC 0.270*** 0.129** 0.181**
(0.080) (0.059) (0.072)
angerC:trueC 0.110 0.089 0.088
(0.091) (0.072) (0.079)
Constant 0.714*** 0.760*** 0.613***
(0.040) (0.030) (0.037)
Observations 13,410 13,409 13,409
Log Likelihood -6,083.703 -5,456.297 -6,890.745
Akaike Inf. Crit. 12,185.410 10,930.590 13,799.490
Bayesian Inf. Crit. 12,252.940 10,998.130 13,867.020
Note: p<0.1; p<0.05; p<0.01

Logistic regression mixed effects model

  belief click share
Predictors Odds Ratios p Odds Ratios p Odds Ratios p
(Intercept) 4.59 <0.001 6.49 <0.001 2.42 <0.001
angerC 0.41 <0.001 0.49 0.001 0.40 <0.001
trueC 8.21 <0.001 2.87 0.015 3.55 0.007
angerC * trueC 1.00 0.995 1.38 0.456 1.36 0.512
Random Effects
σ2 3.29 3.29 3.29
τ00 1.48 respID 2.52 respID 3.70 respID
0.67 headline 0.45 headline 0.53 headline
τ11 0.47 headline.angerC 0.36 headline.angerC 0.45 headline.angerC
ρ01 -0.12 headline -0.12 headline -0.22 headline
ICC 0.41 0.48 0.57
N 1341 respID 1341 respID 1341 respID
10 headline 10 headline 10 headline
Observations 13410 13409 13409
Marginal R2 / Conditional R2 0.183 / 0.521 0.046 / 0.508 0.058 / 0.598

Fear

Linear regression mixed effects model

Dependent variable:
belief click share
(1) (2) (3)
fearC 0.073** 0.078*** 0.099***
(0.031) (0.022) (0.034)
trueC 0.226*** 0.090* 0.163***
(0.058) (0.048) (0.061)
fearC:trueC 0.016 0.024 0.077
(0.061) (0.045) (0.067)
Constant 0.793*** 0.826*** 0.700***
(0.029) (0.025) (0.031)
Observations 13,410 13,409 13,409
Log Likelihood -6,121.023 -5,483.299 -6,906.449
Akaike Inf. Crit. 12,260.050 10,984.600 13,830.900
Bayesian Inf. Crit. 12,327.580 11,052.130 13,898.430
Note: p<0.1; p<0.05; p<0.01

Logistic regression mixed effects model

  belief click share
Predictors Odds Ratios p Odds Ratios p Odds Ratios p
(Intercept) 8.83 <0.001 11.23 <0.001 4.44 <0.001
fearC 2.26 <0.001 2.24 <0.001 2.13 <0.001
trueC 10.08 <0.001 2.85 0.011 3.81 0.002
fearC * trueC 2.21 0.016 1.86 0.063 2.17 0.035
Random Effects
σ2 3.29 3.29 3.29
τ00 1.51 respID 2.53 respID 3.69 respID
0.55 headline 0.39 headline 0.43 headline
τ11 0.15 headline.fearC 0.16 headline.fearC 0.25 headline.fearC
ρ01 -0.38 headline -0.25 headline -0.59 headline
ICC 0.40 0.48 0.57
N 1341 respID 1341 respID 1341 respID
10 headline 10 headline 10 headline
Observations 13410 13409 13409
Marginal R2 / Conditional R2 0.196 / 0.516 0.054 / 0.505 0.060 / 0.594

Happiness

Linear regression mixed effects model

Dependent variable:
belief click share
belief click share
(1) (2) (3)
happyC 0.155*** 0.108*** 0.158***
(0.031) (0.025) (0.026)
trueC 0.191*** 0.077 0.109*
(0.057) (0.049) (0.062)
happyC:trueC -0.196*** -0.088* -0.189***
(0.062) (0.049) (0.052)
Constant 0.802*** 0.826*** 0.705***
(0.029) (0.025) (0.032)
Observations 13,410 13,409 13,409
Log Likelihood -5,853.160 -5,393.049 -6,715.281
Akaike Inf. Crit. 11,724.320 10,804.100 13,448.560
Bayesian Inf. Crit. 11,791.850 10,871.630 13,516.090
Note: p<0.1; p<0.05; p<0.01

Logistic mixed effects model

  belief click share
Predictors Odds Ratios p Odds Ratios p Odds Ratios p
(Intercept) 9.15 <0.001 12.28 <0.001 5.11 <0.001
happyC 3.21 <0.001 3.19 <0.001 3.71 <0.001
trueC 7.07 <0.001 2.25 0.066 2.54 0.057
happyC * trueC 0.48 0.031 0.52 0.121 0.32 0.001
Random Effects
σ2 3.29 3.29 3.29
τ00 1.42 respID 2.55 respID 3.90 respID
0.57 headline 0.45 headline 0.58 headline
τ11 0.17 headline.happyC 0.32 headline.happyC 0.17 headline.happyC
ρ01 -0.70 headline 0.04 headline -0.10 headline
ICC 0.40 0.48 0.58
N 1341 respID 1341 respID 1341 respID
10 headline 10 headline 10 headline
Observations 13410 13409 13409
Marginal R2 / Conditional R2 0.195 / 0.514 0.066 / 0.517 0.082 / 0.614

Sadness

Linear regression mixed effects model

Dependent variable:
belief click share
belief click share
(1) (2) (3)
sadC 0.020 0.018 0.002
(0.024) (0.025) (0.029)
trueC 0.232*** 0.105* 0.173**
(0.072) (0.056) (0.073)
sadC:trueC 0.042 0.057 0.092
(0.048) (0.051) (0.057)
Constant 0.766*** 0.797*** 0.659***
(0.036) (0.029) (0.037)
Observations 13,410 13,409 13,409
Log Likelihood -6,133.851 -5,493.764 -6,947.877
Akaike Inf. Crit. 12,285.700 11,005.530 13,913.750
Bayesian Inf. Crit. 12,353.240 11,073.060 13,981.290
Note: p<0.1; p<0.05; p<0.01

Logistic mixed effects model

  belief click share
Predictors Odds Ratios p Odds Ratios p Odds Ratios p
(Intercept) 7.00 <0.001 8.54 <0.001 3.28 <0.001
sadC 1.53 0.007 1.28 0.177 1.04 0.803
trueC 9.56 <0.001 2.98 0.016 3.81 0.007
sadC * trueC 2.49 0.004 2.12 0.038 2.03 0.025
Random Effects
σ2 3.29 3.29 3.29
τ00 1.48 respID 2.49 respID 3.68 respID
0.80 headline 0.50 headline 0.61 headline
τ11 0.18 headline.sadC 0.26 headline.sadC 0.19 headline.sadC
ρ01 0.47 headline 0.08 headline -0.26 headline
ICC 0.40 0.48 0.57
N 1341 respID 1341 respID 1341 respID
10 headline 10 headline 10 headline
Observations 13410 13409 13409
Marginal R2 / Conditional R2 0.182 / 0.513 0.042 / 0.501 0.045 / 0.591

Surprise

Linear regression mixed effects model

Model with only random intercepts due to singularity.

Dependent variable:
belief click share
belief click share
(1) (2) (3)
surpriseC 0.016** 0.090*** 0.061***
(0.008) (0.007) (0.008)
trueC 0.214*** 0.098 0.144*
(0.077) (0.067) (0.086)
surpriseC:trueC -0.052*** -0.022 -0.038**
(0.015) (0.014) (0.016)
Constant 0.767*** 0.817*** 0.678***
(0.039) (0.034) (0.043)
Observations 13,410 13,409 13,409
Log Likelihood -6,144.651 -5,439.658 -6,937.499
Akaike Inf. Crit. 12,303.300 10,893.320 13,889.000
Bayesian Inf. Crit. 12,355.830 10,945.840 13,941.520
Note: p<0.1; p<0.05; p<0.01

Logit mixed effects model

  belief click share
Predictors Odds Ratios p Odds Ratios p Odds Ratios p
(Intercept) 6.63 <0.001 10.25 <0.001 3.73 <0.001
surpriseC 1.07 0.286 2.26 <0.001 1.50 <0.001
trueC 7.62 <0.001 2.88 0.038 3.23 0.036
surpriseC * trueC 0.78 0.044 1.15 0.289 0.94 0.599
Random Effects
σ2 3.29 3.29 3.29
τ00 1.48 respID 2.37 respID 3.54 respID
0.79 headline 0.64 headline 0.78 headline
ICC 0.41 0.48 0.57
N 1341 respID 1341 respID 1341 respID
10 headline 10 headline 10 headline
Observations 13410 13409 13409
Marginal R2 / Conditional R2 0.162 / 0.504 0.050 / 0.504 0.044 / 0.586

Disgust

Linear regression mixed effects model

Only random slope for headline, otherwise singular for belief logit model.

Dependent variable:
belief click share
belief click share
(1) (2) (3)
disgustC -0.137*** -0.147*** -0.168***
(0.042) (0.034) (0.039)
trueC 0.250*** 0.132** 0.177**
(0.077) (0.058) (0.077)
disgustC:trueC 0.081 0.110 0.086
(0.084) (0.068) (0.078)
Constant 0.706*** 0.739*** 0.599***
(0.039) (0.029) (0.039)
Observations 13,410 13,409 13,409
Log Likelihood -6,091.117 -5,417.688 -6,869.702
Akaike Inf. Crit. 12,200.230 10,853.380 13,757.400
Bayesian Inf. Crit. 12,267.770 10,920.910 13,824.940
Note: p<0.1; p<0.05; p<0.01

Logit mixed effects model

  belief click share
Predictors Odds Ratios p Odds Ratios p Odds Ratios p
(Intercept) 4.27 <0.001 5.72 <0.001 2.23 0.002
disgustC 0.36 <0.001 0.35 <0.001 0.34 <0.001
trueC 6.94 <0.001 2.88 0.015 3.50 0.014
disgustC * trueC 0.79 0.633 1.59 0.269 1.42 0.440
Random Effects
σ2 3.29 3.29 3.29
τ00 1.51 respID 2.60 respID 3.70 respID
0.59 headline 0.44 headline 0.63 headline
τ11 0.44 headline.disgustC 0.29 headline.disgustC 0.39 headline.disgustC
ρ01 -0.24 headline -0.03 headline 0.10 headline
ICC 0.41 0.49 0.57
N 1341 respID 1341 respID 1341 respID
10 headline 10 headline 10 headline
Observations 13410 13409 13409
Marginal R2 / Conditional R2 0.176 / 0.516 0.052 / 0.514 0.059 / 0.597

Anger, Fear, Sadness, Happiness and Belief

Linear mixed effects models with emotion strength

Dependent variable:
belief
(1) (2) (3) (4) (5) (6)
emostrengthanger2 -0.021**
(0.009)
emostrengthfear2 0.012**
(0.006)
emostrengthsad2 0.004
(0.004)
emostrengthhappy2 0.028***
(0.005)
emostrengthsurprise2 0.005***
(0.001)
emostrengthdisgust2 -0.022***
(0.008)
trueC 0.217*** 0.218*** 0.212*** 0.287*** 0.242*** 0.211***
(0.080) (0.079) (0.079) (0.084) (0.077) (0.078)
emostrengthanger2:trueC 0.021
(0.017)
emostrengthfear2:trueC 0.004
(0.011)
emostrengthsad2:trueC 0.008
(0.009)
emostrengthhappy2:trueC -0.032***
(0.010)
emostrengthsurprise2:trueC -0.010***
(0.003)
emostrengthdisgust2:trueC 0.017
(0.017)
Constant 0.772*** 0.757*** 0.756*** 0.725*** 0.756*** 0.772***
(0.040) (0.039) (0.040) (0.042) (0.039) (0.039)
Observations 13,410 13,410 13,410 13,410 13,410 13,410
Log Likelihood -6,094.351 -6,124.775 -6,136.415 -5,846.123 -6,141.368 -6,103.722
Akaike Inf. Crit. 12,206.700 12,267.550 12,290.830 11,710.250 12,296.740 12,225.440
Bayesian Inf. Crit. 12,274.240 12,335.080 12,358.360 11,777.780 12,349.260 12,292.980
Note: p<0.1; p<0.05; p<0.01

Main results interacted with partisanship

There is no significant interaction between identifying with a party “party_close” and veracity or the triple interaction with the emotion for models where the outcome is belief. For models predicting intention to share in the table below, the triple interaction is not significant but veracity[true] x party_close is greater than 1 and statistically significant in the happy and fear models, indicating that individuals who identify with a party are more likely to share true than false info, compare to people who do not identify with any party. Overall, partisans are more likely to share stimuli than non-partisans. [below are logit models]

  share share share
Predictors Odds Ratios p Odds Ratios p Odds Ratios p
(Intercept) 4.48 <0.001 2.74 <0.001 3.62 <0.001
happyC 4.24 <0.001
trueC 2.53 0.058 3.66 0.010 3.79 0.002
party_close 1.84 <0.001 2.30 <0.001 2.53 <0.001
happyC * trueC 0.33 0.002
happyC * party_close 0.52 <0.001
trueC * party_close 1.12 0.482 1.38 0.037 1.11 0.585
(happyC * trueC) *
party_close
0.72 0.335
sadC 1.04 0.789
sadC * trueC 1.99 0.031
sadC * party_close 0.95 0.747
(sadC * trueC) *
party_close
1.16 0.628
fearC 2.02 <0.001
fearC * trueC 2.33 0.026
fearC * party_close 1.25 0.273
(fearC * trueC) *
party_close
0.68 0.325
Random Effects
σ2 3.29 3.29 3.29
τ00 3.74 respID 3.51 respID 3.54 respID
0.57 headline 0.61 headline 0.44 headline
τ11 0.18 headline.happyC 0.18 headline.sadC 0.25 headline.fearC
ρ01 -0.12 headline -0.29 headline -0.57 headline
ICC 0.57 0.56 0.56
N 1320 respID 1320 respID 1320 respID
10 headline 10 headline 10 headline
Observations 13200 13200 13200
Marginal R2 / Conditional R2 0.099 / 0.614 0.064 / 0.590 0.079 / 0.594

The below table interacts main models with APC (ruling party partisanship) - among individuals who say they identify with a political party. A majority of respondents (60%) do not say they feel close to any political party, which are used in the table above. [below are logit models w/ only random intercepts]

  belief belief belief
Predictors Odds Ratios p Odds Ratios p Odds Ratios p
(Intercept) 10.98 <0.001 9.70 <0.001 12.00 <0.001
happyC 2.54 <0.001
trueC 9.00 <0.001 13.34 <0.001 12.63 <0.001
APC 1.54 0.097 1.01 0.980 1.16 0.587
happyC * trueC 0.28 0.009
happyC * APC 1.96 0.097
trueC * APC 0.78 0.521 0.59 0.081 0.72 0.423
(happyC * trueC) * APC 3.27 0.135
sadC 1.49 0.082
sadC * trueC 1.48 0.378
sadC * APC 0.68 0.226
(sadC * trueC) * APC 1.89 0.299
fearC 2.38 0.003
fearC * trueC 1.24 0.704
fearC * APC 1.02 0.959
(fearC * trueC) * APC 3.03 0.182
Random Effects
σ2 3.29 3.29 3.29
τ00 2.01 respID 2.19 respID 2.22 respID
0.64 headline 0.62 headline 0.58 headline
ICC 0.45 0.46 0.46
N 306 respID 306 respID 306 respID
10 headline 10 headline 10 headline
Observations 3060 3060 3060
Marginal R2 / Conditional R2 0.206 / 0.561 0.179 / 0.558 0.202 / 0.570
  belief click share
Predictors Odds Ratios p Odds Ratios p Odds Ratios p
(Intercept) 8.87 <0.001 11.79 <0.001 4.92 <0.001
trueC 12.97 <0.001 3.39 0.004 3.82 0.006
APC 1.17 0.468 2.16 0.007 1.90 0.011
trueC * APC 0.47 0.001 1.16 0.561 0.96 0.841
Random Effects
σ2 3.29 3.29 3.29
τ00 2.20 respID 3.78 respID 3.70 respID
0.68 headline 0.38 headline 0.54 headline
ICC 0.47 0.56 0.56
N 306 respID 306 respID 306 respID
10 headline 10 headline 10 headline
Observations 3060 3060 3060
Marginal R2 / Conditional R2 0.170 / 0.557 0.071 / 0.589 0.067 / 0.592

Main linear models with controls

Main results for neutral, fear, sadness, and happiness using linear mixed effect regressions - controlling for gender, age, and education level. Belief regression controlling for gender, age, education level.[All models have both headline and respondent random intercepts and random slopes for the emotion wrt headline.]

Dependent variable:
belief
(1) (2) (3) (4) (5) (6) (7)
neutralC -0.174***
(0.029)
fearC 0.074**
(0.031)
sadC 0.020
(0.024)
angerC -0.120***
(0.047)
happyC 0.152***
(0.032)
surpriseC 0.017**
(0.008)
disgustC -0.139***
(0.043)
trueC 0.271*** 0.228*** 0.234*** 0.274*** 0.192*** 0.215*** 0.251***
(0.090) (0.058) (0.071) (0.080) (0.057) (0.077) (0.076)
age -0.001 -0.001 -0.001 -0.0004 -0.0005 -0.001 -0.0004
(0.001) (0.001) (0.001) (0.001) (0.001) (0.001) (0.001)
gender -0.010 0.00004 0.001 0.001 -0.001 -0.001 0.003
(0.011) (0.011) (0.011) (0.011) (0.011) (0.011) (0.011)
edu -0.002 -0.002 -0.002 -0.002 -0.002 -0.002 -0.002
(0.005) (0.005) (0.005) (0.005) (0.005) (0.005) (0.005)
neutralC:trueC 0.125**
(0.057)
fearC:trueC 0.014
(0.063)
sadC:trueC 0.041
(0.048)
angerC:trueC 0.113
(0.093)
happyC:trueC -0.200***
(0.063)
surpriseC:trueC -0.055***
(0.015)
disgustC:trueC 0.078
(0.086)
Constant 0.744*** 0.829*** 0.797*** 0.743*** 0.835*** 0.798*** 0.728***
(0.058) (0.048) (0.052) (0.055) (0.046) (0.054) (0.054)
Observations 13,190 13,190 13,190 13,190 13,190 13,190 13,190
Log Likelihood -5,884.218 -6,011.762 -6,026.376 -5,977.404 -5,746.957 -6,035.761 -5,983.126
Akaike Inf. Crit. 11,792.430 12,047.520 12,076.750 11,978.810 11,517.910 12,091.520 11,990.250
Bayesian Inf. Crit. 11,882.280 12,137.370 12,166.600 12,068.650 11,607.760 12,166.390 12,080.100
Note: p<0.1; p<0.05; p<0.01

Leave one out analysis

The below graphs plot the coefficients on the interaction terms (emotion x veracity) for the linear mixed effect models predicting belief. These graphs plot estimates on the y axis. The points represent the iterated estimates when 1 out of the 10 headlines is removed from the analysis (to ensure that the results are not being driven by a few headlines). The x-axis indicates which headline is left out.

All emotions in single model

Linear regression model:

Dependent variable:
belief click share
(1) (2) (3)
trueC 0.210*** 0.076* 0.126**
(0.065) (0.044) (0.056)
angerC -0.020* -0.013 -0.035***
(0.010) (0.010) (0.011)
disgustC -0.032*** -0.070*** -0.071***
(0.011) (0.010) (0.012)
fearC 0.101*** 0.110*** 0.150***
(0.010) (0.009) (0.010)
happyC 0.228*** 0.179*** 0.245***
(0.010) (0.010) (0.011)
sadC 0.094*** 0.100*** 0.103***
(0.009) (0.008) (0.009)
surpriseC 0.063*** 0.128*** 0.110***
(0.008) (0.008) (0.009)
Constant 0.886*** 0.911*** 0.801***
(0.034) (0.025) (0.031)
Observations 13,410 13,409 13,409
Log Likelihood -5,836.878 -5,154.609 -6,534.496
Akaike Inf. Crit. 11,695.760 10,331.220 13,090.990
Bayesian Inf. Crit. 11,778.300 10,413.760 13,173.530
Note: p<0.1; p<0.05; p<0.01

Secondary Analysis

Emotions and time spent on belief question

with emotion absence emo. p.value test
11.75 9.38 0.00 neutral
8.89 9.90 0.00 sad
8.65 9.84 0.00 fear
9.03 9.81 0.02 happy
10.46 9.52 0.27 anger
9.84 9.57 0.50 surprise
9.66 9.65 0.99 disgust

Emotion Regulation

Reappraisal

Logit
  belief click share
Predictors Odds Ratios p Odds Ratios p Odds Ratios p
(Intercept) 2.27 0.047 1.24 0.613 0.38 0.044
erq_reapp 1.20 <0.001 1.40 <0.001 1.47 <0.001
trueC 38.42 <0.001 3.82 0.029 10.03 <0.001
erq_reapp * trueC 0.77 <0.001 0.94 0.294 0.82 0.001
Random Effects
σ2 3.29 3.29 3.29
τ00 1.49 respID 2.28 respID 3.32 respID
0.78 headline 0.64 headline 0.75 headline
ICC 0.41 0.47 0.55
N 1228 respID 1228 respID 1228 respID
10 headline 10 headline 10 headline
Observations 12280 12280 12280
Marginal R2 / Conditional R2 0.172 / 0.510 0.049 / 0.496 0.058 / 0.579
Linear
  belief click share
Predictors Estimates p Estimates p Estimates p
(Intercept) 0.60 <0.001 0.59 <0.001 0.39 <0.001
erq_reapp 0.03 <0.001 0.04 <0.001 0.05 <0.001
trueC 0.53 <0.001 0.22 0.007 0.39 <0.001
erq_reapp * trueC -0.05 <0.001 -0.02 0.003 -0.04 <0.001
Random Effects
σ2 0.13 0.12 0.14
τ00 0.02 respID 0.03 respID 0.06 respID
0.01 headline 0.01 headline 0.02 headline
ICC 0.22 0.27 0.36
N 1228 respID 1228 respID 1228 respID
10 headline 10 headline 10 headline
Observations 12280 12280 12280
Marginal R2 / Conditional R2 0.077 / 0.283 0.020 / 0.283 0.034 / 0.377

Suppression

Logit
  belief click share
Predictors Odds Ratios p Odds Ratios p Odds Ratios p
(Intercept) 4.31 <0.001 4.99 <0.001 0.84 0.615
erq_supp 1.10 0.002 1.13 0.001 1.37 <0.001
trueC 19.49 <0.001 3.37 0.023 4.67 0.007
erq_supp * trueC 0.83 <0.001 0.95 0.172 0.92 0.021
Random Effects
σ2 3.29 3.29 3.29
τ00 1.50 respID 2.38 respID 3.29 respID
0.79 headline 0.64 headline 0.75 headline
ICC 0.41 0.48 0.55
N 1248 respID 1248 respID 1248 respID
10 headline 10 headline 10 headline
Observations 12480 12480 12480
Marginal R2 / Conditional R2 0.173 / 0.512 0.041 / 0.500 0.067 / 0.582
Linear
  belief click share
Predictors Estimates p Estimates p Estimates p
(Intercept) 0.70 <0.001 0.75 <0.001 0.49 <0.001
erq_supp 0.02 <0.001 0.01 0.007 0.04 <0.001
trueC 0.37 <0.001 0.14 0.045 0.26 0.004
erq_supp * trueC -0.03 <0.001 -0.01 0.015 -0.02 <0.001
Random Effects
σ2 0.13 0.12 0.14
τ00 0.02 respID 0.03 respID 0.06 respID
0.01 headline 0.01 headline 0.02 headline
ICC 0.22 0.27 0.35
N 1248 respID 1248 respID 1248 respID
10 headline 10 headline 10 headline
Observations 12480 12480 12480
Marginal R2 / Conditional R2 0.076 / 0.283 0.015 / 0.284 0.039 / 0.379

Cognitive Reflection

% Table created by stargazer v.5.2.2 by Marek Hlavac, Harvard University. E-mail: hlavac at fas.harvard.edu % Date and time: Thu, Dec 17, 2020 - 14:46:07

Correlations with CRT

Dependent variable:
crt_index
(1) (2) (3) (4) (5)
erq_reapp -0.243***
(0.054)
erq_supp -0.156***
(0.034)
age -0.036*** -0.035*** -0.035*** -0.035*** -0.036***
(0.007) (0.007) (0.007) (0.007) (0.007)
factor(gender)1 0.247** 0.174* 0.183* 0.158 0.187*
(0.100) (0.100) (0.100) (0.099) (0.100)
edu 0.065 0.071* 0.070* 0.075* 0.070*
(0.041) (0.042) (0.042) (0.041) (0.042)
factor(urban)1 0.231** 0.219* 0.222** 0.215* 0.238**
(0.111) (0.112) (0.112) (0.112) (0.112)
factor(voted)1 -0.205** -0.309*** -0.304*** -0.319*** -0.320***
(0.094) (0.093) (0.093) (0.093) (0.093)
christ 0.178* 0.155 -0.167 0.070
(0.105) (0.105) (0.325) (0.104)
religiosity -0.058 -0.062 -0.135 -0.058
(0.045) (0.045) (0.082) (0.045)
christ:religiosity 0.103
(0.098)
Constant 4.297*** 2.235*** 2.464*** 2.318*** 2.122***
(0.464) (0.365) (0.426) (0.361) (0.338)
Observations 1,172 1,193 1,193 1,193 1,210
R2 0.082 0.040 0.041 0.038 0.039
Adjusted R2 0.075 0.034 0.034 0.033 0.034
Residual Std. Error 1.500 (df = 1162) 1.529 (df = 1185) 1.529 (df = 1184) 1.530 (df = 1186) 1.543 (df = 1203)
F Statistic 11.596*** (df = 9; 1162) 7.027*** (df = 7; 1185) 6.286*** (df = 8; 1184) 7.831*** (df = 6; 1186) 8.078*** (df = 6; 1203)
Note: p<0.1; p<0.05; p<0.01

Correlations with discernment

Dependent variable:
discern
(1) (2) (3) (4) (5) (6) (7)
erq_reapp -0.274***
(0.054)
erq_supp -0.181***
(0.033)
crt_index 0.192*** 0.191*** 0.193***
(0.029) (0.029) (0.029)
age 0.013* 0.009 0.018** 0.018** 0.012* 0.012* 0.018**
(0.007) (0.007) (0.007) (0.007) (0.007) (0.007) (0.007)
factor(gender)1 0.153 0.252** 0.122 0.134 0.155 0.164* 0.121
(0.102) (0.102) (0.102) (0.101) (0.098) (0.097) (0.102)
edu 0.061 0.055 0.049 0.046 0.052 0.050 0.047
(0.042) (0.042) (0.042) (0.042) (0.040) (0.040) (0.042)
factor(urban)1 0.269** 0.279** 0.212* 0.216* 0.267** 0.269** 0.223**
(0.113) (0.112) (0.113) (0.113) (0.109) (0.109) (0.113)
factor(APC_party)1 -0.376** -0.370** -0.363** -0.344** -0.395*** -0.383***
(0.146) (0.144) (0.144) (0.143) (0.142) (0.141)
factor(party_close)1 -0.120
(0.108)
factor(voted)1 0.043 0.049 0.026 0.031 -0.017 -0.013 0.007
(0.097) (0.097) (0.096) (0.096) (0.094) (0.094) (0.097)
factor(christ)1 -0.027 -0.121 -0.121 -0.084 -0.088
(0.108) (0.106) (0.106) (0.105) (0.106)
religiosity -0.067 -0.101** -0.081* -0.085* -0.097** -0.099** -0.082*
(0.045) (0.045) (0.045) (0.045) (0.044) (0.044) (0.045)
someuse 0.039 0.030 0.040* 0.040 0.036 0.036 0.040
(0.025) (0.024) (0.025) (0.025) (0.023) (0.023) (0.025)
Constant 6.308*** 5.886*** 4.476*** 4.416*** 5.012*** 4.962*** 4.469***
(0.516) (0.469) (0.447) (0.444) (0.427) (0.422) (0.448)
Observations 1,192 1,211 1,193 1,193 1,276 1,276 1,193
R2 0.048 0.051 0.062 0.061 0.028 0.028 0.058
Adjusted R2 0.040 0.043 0.054 0.054 0.021 0.021 0.050
Residual Std. Error 1.542 (df = 1181) 1.538 (df = 1200) 1.532 (df = 1182) 1.532 (df = 1183) 1.545 (df = 1266) 1.544 (df = 1267) 1.535 (df = 1182)
F Statistic 5.922*** (df = 10; 1181) 6.405*** (df = 10; 1200) 7.833*** (df = 10; 1182) 8.558*** (df = 9; 1183) 4.057*** (df = 9; 1266) 4.486*** (df = 8; 1267) 7.290*** (df = 10; 1182)
Note: p<0.1; p<0.05; p<0.01
Correlates of Discernment

Correlates of Discernment

Are emotion regulation and suppression correlated with felt emotions?

Dependent variable:
emo.index
(1) (2)
erq_reapp -0.095
(0.061)
erq_supp -0.055
(0.038)
age -0.015* -0.016**
(0.008) (0.008)
factor(gender)1 -0.405*** -0.380***
(0.114) (0.114)
edu 0.016 0.019
(0.048) (0.047)
factor(urban)1 0.031 0.045
(0.128) (0.127)
factor(voted)1 -0.494*** -0.468***
(0.107) (0.107)
factor(christ)1 0.108 0.097
(0.122) (0.120)
religiosity -0.067 -0.072
(0.051) (0.051)
Constant 2.524*** 2.225***
(0.522) (0.456)
Observations 1,192 1,211
R2 0.050 0.048
Adjusted R2 0.044 0.042
Residual Std. Error 1.748 (df = 1183) 1.751 (df = 1202)
F Statistic 7.795*** (df = 8; 1183) 7.599*** (df = 8; 1202)
Note: p<0.1; p<0.05; p<0.01

Does being asked about emotions/belief first matter?

  belief click share
Predictors Odds Ratios p Odds Ratios p Odds Ratios p
(Intercept) 2.15 0.057 5.21 <0.001 1.80 0.142
veracity [true] 7.68 <0.001 2.72 0.042 3.04 0.045
A_or_B [B] 1.18 0.059 1.07 0.529 1.19 0.156
veracity [true] * A_or_B
[B]
1.09 0.428 0.84 0.083 1.04 0.692
Random Effects
σ2 3.29 3.29 3.29
τ00 1.49 respID 2.53 respID 3.66 respID
0.80 headline 0.60 headline 0.76 headline
ICC 0.41 0.49 0.57
N 1341 respID 1341 respID 1341 respID
10 headline 10 headline 10 headline
Observations 13410 13409 13409
Marginal R2 / Conditional R2 0.164 / 0.506 0.032 / 0.503 0.041 / 0.591
  belief click share
Predictors Estimates p Estimates p Estimates p
(Intercept) 0.64 <0.001 0.84 <0.001 0.70 <0.001
A_or_B [B] 0.03 0.236 0.01 0.502 0.01 0.549
Observations 1341 1341 1341
R2 / R2 adjusted 0.001 / 0.000 0.000 / -0.000 0.000 / -0.000

Are people who got more headlines WRONG more likely to seek information after debrief?

No, there does not seem to be any strong predictors of clicking for more information at the end of the survey. But overall, very few people did this – 66 people clicked on Africa check links and 34 people clicked on WHO links.

Dependent variable:
click_WHO click_africacheck click_WHOwhatsapp click_ACwhatsapp click.any
(1) (2) (3) (4) (5)
emo.index 0.0002 0.001 -0.001 -0.002 -0.003
(0.001) (0.001) (0.003) (0.003) (0.004)
corr.index -0.0004 -0.001 -0.001 -0.001 -0.001
(0.001) (0.001) (0.003) (0.004) (0.005)
age -0.0002 -0.0001 -0.0003 -0.0004 -0.001
(0.0003) (0.0004) (0.001) (0.001) (0.001)
factor(gender)1 -0.005 0.009* -0.007 0.036*** 0.034**
(0.004) (0.005) (0.010) (0.014) (0.016)
edu -0.001 -0.002 -0.003 0.008 0.007
(0.002) (0.002) (0.004) (0.006) (0.007)
factor(urban)1 0.005 -0.0004 -0.018 -0.029* -0.042**
(0.004) (0.006) (0.011) (0.015) (0.018)
factor(voted)1 -0.004 0.008 -0.012 0.001 -0.009
(0.003) (0.005) (0.009) (0.013) (0.015)
factor(christ)1 -0.001 0.006 0.002 0.010 0.003
(0.004) (0.006) (0.011) (0.014) (0.017)
religiosity 0.001 0.003 0.008* 0.006 0.014*
(0.002) (0.002) (0.004) (0.006) (0.007)
Constant 0.018 0.009 0.066* -0.035 0.028
(0.015) (0.021) (0.040) (0.054) (0.064)
Observations 1,219 1,219 1,219 1,219 1,276
R2 0.007 0.009 0.009 0.013 0.014
Adjusted R2 -0.0001 0.002 0.002 0.006 0.007
Residual Std. Error 0.057 (df = 1209) 0.081 (df = 1209) 0.155 (df = 1209) 0.211 (df = 1209) 0.253 (df = 1266)
F Statistic 0.981 (df = 9; 1209) 1.246 (df = 9; 1209) 1.250 (df = 9; 1209) 1.755* (df = 9; 1209) 1.992** (df = 9; 1266)
Note: p<0.1; p<0.05; p<0.01

Are people who felt more emotions (and particular kinds) more likely to seek information?

Dependent variable:
click_WHO click_africacheck click_WHOwhatsapp click_ACwhatsapp click.any
(1) (2) (3) (4) (5)
anger.ind 0.002 -0.001 0.005 -0.001 0.003
(0.001) (0.002) (0.003) (0.004) (0.005)
fear.ind -0.001 -0.003 0.002 -0.005 -0.006
(0.001) (0.002) (0.003) (0.005) (0.005)
sad.ind 0.002 -0.0005 -0.004 0.002 0.001
(0.001) (0.001) (0.003) (0.004) (0.004)
happy.ind -0.0003 -0.002 0.002 0.0001 0.002
(0.001) (0.002) (0.003) (0.004) (0.005)
surprise.ind -0.0004 0.001 -0.001 0.009*** 0.007*
(0.001) (0.001) (0.002) (0.003) (0.004)
disgust.ind -0.002 0.005** -0.0001 -0.0004 0.001
(0.001) (0.002) (0.004) (0.005) (0.006)
corr.index -0.001 -0.002 -0.001 0.001 -0.0003
(0.001) (0.002) (0.003) (0.004) (0.005)
age -0.0002 -0.0001 -0.0003 -0.0005 -0.001
(0.0003) (0.0004) (0.001) (0.001) (0.001)
factor(gender)1 -0.004 0.007 -0.006 0.033** 0.033**
(0.004) (0.005) (0.010) (0.014) (0.016)
edu -0.001 -0.002 -0.003 0.008 0.007
(0.002) (0.002) (0.004) (0.006) (0.007)
factor(urban)1 0.004 0.0001 -0.018 -0.030* -0.044**
(0.004) (0.006) (0.011) (0.015) (0.018)
factor(voted)1 -0.004 0.008 -0.012 -0.0002 -0.009
(0.003) (0.005) (0.009) (0.013) (0.015)
factor(christ)1 -0.001 0.007 0.003 0.010 0.004
(0.004) (0.006) (0.011) (0.014) (0.017)
religiosity 0.001 0.003 0.008* 0.006 0.014**
(0.002) (0.002) (0.004) (0.006) (0.007)
Constant 0.016 0.014 0.059 -0.059 -0.004
(0.015) (0.022) (0.042) (0.057) (0.067)
Observations 1,219 1,219 1,219 1,219 1,276
R2 0.012 0.016 0.014 0.020 0.017
Adjusted R2 0.001 0.005 0.002 0.008 0.006
Residual Std. Error 0.057 (df = 1204) 0.081 (df = 1204) 0.155 (df = 1204) 0.210 (df = 1204) 0.253 (df = 1261)
F Statistic 1.053 (df = 14; 1204) 1.402 (df = 14; 1204) 1.193 (df = 14; 1204) 1.736** (df = 14; 1204) 1.557* (df = 14; 1261)
Note: p<0.1; p<0.05; p<0.01

Headline Descriptives

headline false_1 false_2 false_3 false_4 false_5 true_1 true_2 true_3 true_4 true_5
fear.mean 0.03 0.15 0.04 0.05 0.09 0.21 0.03 0.39 0.13 0.46
fear.sd 0.18 0.35 0.21 0.21 0.29 0.41 0.18 0.49 0.34 0.50
anger.mean 0.04 0.34 0.05 0.19 0.21 0.04 0.28 0.05 0.06 0.13
anger.sd 0.20 0.47 0.22 0.39 0.41 0.19 0.45 0.21 0.25 0.34
sad.mean 0.05 0.20 0.04 0.10 0.16 0.53 0.13 0.46 0.23 0.51
sad.sd 0.21 0.40 0.20 0.30 0.36 0.50 0.34 0.50 0.42 0.50
happy.mean 0.51 0.04 0.50 0.14 0.09 0.03 0.06 0.14 0.49 0.04
happy.sd 0.50 0.19 0.50 0.35 0.28 0.17 0.24 0.35 0.50 0.19
surprise.mean 0.43 0.29 0.42 0.42 0.28 0.32 0.33 0.12 0.11 0.17
surprise.sd 0.50 0.45 0.49 0.49 0.45 0.47 0.47 0.32 0.31 0.38
disgust.mean 0.04 0.21 0.05 0.17 0.27 0.03 0.28 0.03 0.03 0.05
disgust.sd 0.20 0.41 0.21 0.37 0.44 0.17 0.45 0.16 0.17 0.22
neutral.mean 0.11 0.11 0.11 0.14 0.16 0.10 0.14 0.10 0.14 0.05
neutral.sd 0.31 0.31 0.31 0.35 0.36 0.30 0.35 0.30 0.34 0.21
belief.mean 0.66 0.55 0.68 0.67 0.72 0.94 0.59 0.96 0.94 0.96
belief.sd 0.47 0.50 0.47 0.47 0.45 0.23 0.49 0.20 0.25 0.20
click.mean 0.84 0.73 0.83 0.72 0.65 0.86 0.63 0.92 0.87 0.94
click.sd 0.36 0.44 0.38 0.45 0.48 0.35 0.48 0.28 0.34 0.24
share.mean 0.71 0.51 0.68 0.55 0.53 0.71 0.46 0.86 0.79 0.87
share.sd 0.45 0.50 0.47 0.50 0.50 0.45 0.50 0.35 0.40 0.34

Table presenting the prevalence of felt emotions across all headlines – disaggregated by true/false.

Stimuli

Images of the 10 headlines – 5 true and 5 false and included below.

False 1

False 1

False 2

False 2

False 3

False 3

False 4

False 4

False 5

False 5

True 1

True 1

True 2

True 2

True 3

True 3

True 4

True 4

True 5

True 5

Sample Descriptives

Descriptive statistics of study respondents
Statistic N Mean St. Dev. Min Pctl(25) Median Pctl(75) Max
Female 1,325 0.32 0.47 0.00 0.00 0.00 1.00 1.00
Age 1,321 27.54 7.05 18.00 22.00 26.00 31.00 62.00
Some university 1,321 0.76 0.43 0.00 1.00 1.00 1.00 1.00
Urban 1,297 0.80 0.40 0.00 1.00 1.00 1.00 1.00
Employed 1,316 0.49 0.50 0.00 0.00 0.00 1.00 1.00
Christian 1,316 0.76 0.43 0.00 1.00 1.00 1.00 1.00
Religiosity 1,298 3.24 0.99 0.00 3.00 4.00 4.00 4.00
Voted 1,318 0.45 0.50 0.00 0.00 0.00 1.00 1.00
APC supporter 306 0.47 0.50 0.00 0.00 0.00 1.00 1.00
Daily FB user 1,314 0.74 0.44 0.00 0.00 1.00 1.00 1.00